1 Introduction

In this evaluation, there are a total of 6 data tables. Evaluation metrics from the OmicsEV package for these data tables are included in this report, beginning with a summary of the data. The sample distribution by class for each data table is shown in the table below.

class diann diaumpire diaumpiredda msfraggerdia msfraggerdiadda tmt
Normal 79 79 79 79 79 79
Tumor 103 103 103 103 103 103

Detailed information for each sample included in all data tables is shown below.

sample class batch order
C3L-01287-N Normal 1 1
C3L-00561-N Normal 1 2
C3L-01603-N Normal 1 3
C3N-00834-N Normal 1 4
C3N-01214-N Normal 1 5
C3N-01261-N Normal 1 6
C3L-00917-N Normal 1 7
C3L-00607-N Normal 1 8
C3N-00194-N Normal 1 9
C3L-00010-N Normal 1 10
C3L-01861-N Normal 1 11
C3N-01646-N Normal 1 12
C3L-01281-N Normal 1 13
C3N-00495-N Normal 1 14
C3N-00831-N Normal 1 15
C3L-00183-N Normal 1 16
C3N-00168-N Normal 1 17
C3L-00369-N Normal 1 18
C3L-00791-N Normal 1 19
C3L-00097-N Normal 1 20
C3L-00004-N Normal 1 21
C3N-00953-N Normal 1 22
C3N-00150-N Normal 1 23
C3N-00244-N Normal 1 24
C3N-01178-N Normal 1 25
C3L-00583-N Normal 1 26
C3L-00088-N Normal 1 27
C3L-00814-N Normal 1 28
C3L-01885-N Normal 1 29
C3L-00908-N Normal 1 30
C3L-00026-N Normal 1 31
C3L-00447-N Normal 1 32
C3L-00416-N Normal 1 33
C3N-00246-N Normal 1 34
C3N-00148-N Normal 1 35
C3N-00646-N Normal 1 36
C3L-00902-N Normal 1 37
C3L-00096-N Normal 1 38
C3L-01836-N Normal 1 39
C3N-00177-N Normal 1 40
C3N-01200-N Normal 1 41
C3L-00011-N Normal 1 42
C3N-01808-N Normal 1 43
C3N-01176-N Normal 1 44
C3L-00103-N Normal 1 45
C3N-01649-N Normal 1 46
C3N-00149-N Normal 1 47
C3L-00581-N Normal 1 48
C3L-01313-N Normal 1 49
C3N-00573-N Normal 1 50
C3L-01302-N Normal 1 51
C3N-01361-N Normal 1 52
C3N-01651-N Normal 1 53
C3L-00360-N Normal 1 54
C3L-00448-N Normal 1 55
C3L-00907-N Normal 1 56
C3L-01286-N Normal 1 57
C3L-01607-N Normal 1 58
C3N-01179-N Normal 1 59
C3L-00606-N Normal 1 60
C3N-01648-N Normal 1 61
C3N-00242-N Normal 1 62
C3L-00418-N Normal 1 63
C3N-00577-N Normal 1 64
C3L-01882-N Normal 1 65
C3N-00733-N Normal 1 66
C3L-00910-N Normal 1 67
C3L-00079-N Normal 1 68
C3N-01522-N Normal 1 69
C3N-01220-N Normal 1 70
C3N-00852-N Normal 1 71
C3N-00320-N Normal 1 72
C3N-00437-N Normal 1 73
C3N-00491-N Normal 1 74
C3N-00317-N Normal 1 75
C3N-00310-N Normal 1 76
C3N-00390-N Normal 1 77
C3N-00312-N Normal 1 78
C3N-00494-N Normal 1 79
C3N-01648-T Tumor 1 80
C3N-01214-T Tumor 1 81
C3N-00646-T Tumor 1 82
C3L-00766-T Tumor 1 83
C3L-00360-T Tumor 1 84
C3L-00790-T Tumor 1 85
C3N-00154-T Tumor 1 86
C3L-00765-T Tumor 1 87
C3N-00150-T Tumor 1 88
C3L-01283-T Tumor 1 89
C3N-00315-T Tumor 1 90
C3N-00177-T Tumor 1 91
C3L-00561-T Tumor 1 92
C3L-00800-T Tumor 1 93
C3L-01882-T Tumor 1 94
C3N-00491-T Tumor 1 95
C3N-00577-T Tumor 1 96
C3L-00079-T Tumor 1 97
C3N-01220-T Tumor 1 98
C3L-01607-T Tumor 1 99
C3N-00953-T Tumor 1 100
C3N-00380-T Tumor 1 101
C3N-00317-T Tumor 1 102
C3N-00305-T Tumor 1 103
C3N-00437-T Tumor 1 104
C3N-01651-T Tumor 1 105
C3L-00908-T Tumor 1 106
C3L-00416-T Tumor 1 107
C3N-00733-T Tumor 1 108
C3L-00610-T Tumor 1 109
C3N-01213-T Tumor 1 110
C3N-01200-T Tumor 1 111
C3N-01361-T Tumor 1 112
C3L-00183-T Tumor 1 113
C3L-01861-T Tumor 1 114
C3L-00917-T Tumor 1 115
C3L-00817-T Tumor 1 116
C3L-00581-T Tumor 1 117
C3L-00004-T Tumor 1 118
C3N-01524-T Tumor 1 119
C3L-01302-T Tumor 1 120
C3L-01836-T Tumor 1 121
C3N-00314-T Tumor 1 122
C3L-01287-T Tumor 1 123
C3N-00149-T Tumor 1 124
C3N-00148-T Tumor 1 125
C3N-00390-T Tumor 1 126
C3L-01885-T Tumor 1 127
C3N-00495-T Tumor 1 128
C3N-01646-T Tumor 1 129
C3L-00813-T Tumor 1 130
C3L-00812-T Tumor 1 131
C3N-00494-T Tumor 1 132
C3N-00573-T Tumor 1 133
C3L-01560-T Tumor 1 134
C3N-01649-T Tumor 1 135
C3L-00792-T Tumor 1 136
C3N-01179-T Tumor 1 137
C3N-00168-T Tumor 1 138
C3L-00607-T Tumor 1 139
C3L-00907-T Tumor 1 140
C3N-01178-T Tumor 1 141
C3L-00902-T Tumor 1 142
C3N-00831-T Tumor 1 143
C3L-00096-T Tumor 1 144
C3N-00194-T Tumor 1 145
C3L-00010-T Tumor 1 146
C3N-00852-T Tumor 1 147
C3N-01522-T Tumor 1 148
C3L-01288-T Tumor 1 149
C3L-00799-T Tumor 1 150
C3L-00088-T Tumor 1 151
C3L-00910-T Tumor 1 152
C3N-01261-T Tumor 1 153
C3L-00606-T Tumor 1 154
C3L-01603-T Tumor 1 155
C3L-01553-T Tumor 1 156
C3L-01281-T Tumor 1 157
C3N-00320-T Tumor 1 158
C3L-01557-T Tumor 1 159
C3L-01352-T Tumor 1 160
C3N-01176-T Tumor 1 161
C3N-00834-T Tumor 1 162
C3L-00418-T Tumor 1 163
C3L-00796-T Tumor 1 164
C3L-00103-T Tumor 1 165
C3L-00097-T Tumor 1 166
C3N-00312-T Tumor 1 167
C3L-00583-T Tumor 1 168
C3L-00814-T Tumor 1 169
C3L-01313-T Tumor 1 170
C3L-00026-T Tumor 1 171
C3L-00447-T Tumor 1 172
C3L-00448-T Tumor 1 173
C3N-01808-T Tumor 1 174
C3L-00011-T Tumor 1 175
C3N-00310-T Tumor 1 176
C3N-00244-T Tumor 1 177
C3L-00791-T Tumor 1 178
C3L-01286-T Tumor 1 179
C3N-00246-T Tumor 1 180
C3N-00242-T Tumor 1 181
C3L-00369-T Tumor 1 182

2 Overview

The table below provides an overview about all the quantitative metrics generated in the evaluation. For each metric, the value of the best data table is highlighted in bold and red. The details for each metric can be found in the corresponding sections below.

metric diann diaumpire diaumpiredda msfraggerdia msfraggerdiadda tmt
#identified features 9414
(0.4618) 		7940
(0.3895) 		8478
(0.4159) 		7973
(0.3911) 		8485
(0.4162) 		12082
(0.5927)
#quantifiable features 6224
(0.3053) 		6722
(0.3297) 		6885
(0.3377) 		6652
(0.3263) 		6755
(0.3314) 		9526
(0.4673)
non_missing_value_ratio 0.9022 0.9255 0.9208 0.9224 0.9190 0.9526
data_dist_similarity 0.9436 0.9601 0.9567 0.9593 0.9592 0.9824
complex_auc 0.7878 0.7950 0.7970 0.7903 0.7961 0.8668
func_auc 0.7886 0.8123 0.8048 0.8015 0.7950 0.8130
gene_wise_cor 0.3820 0.3733 0.3675 0.3683 0.3693 0.4043
sample_wise_cor 0.4025 0.3888 0.3914 0.3867 0.3887 0.4595

The radar plot below summarizes results from the overview table above. To generate the radar plot, each metric is scaled from 0 to 1 such that higher values indicate better data quality if necessary. Scaled values are in parentheses in the table.

3 Data depth

3.1 Study-wise

The table below shows the number of identified and quantified proteins or genes for each data table. Identified proteins or genes are those with a measurement in any sample in a data table whereas quantified proteins or genes are those that remain after filtering out those with missing values in more than 50% of the samples in a data table. The values in parentheses are the percentage of proteins or genes identified or quantified based on the total number of proteins or genes (20386) in the study species.

data table #identified features #quantifiable features
diann 9414
(46.18%) 		6224
(30.53%)
diaumpire 7940
(38.95%) 		6722
(32.97%)
diaumpiredda 8478
(41.59%) 		6885
(33.77%)
msfraggerdia 7973
(39.11%) 		6652
(32.63%)
msfraggerdiadda 8485
(41.62%) 		6755
(33.14%)
tmt 12082
(59.27%) 		9526
(46.73%)

The upset chart below shows overlap between proteins or genes identified in each data table. Numbers of proteins or genes commonly identified in different combinations of data tables are indicated in the top bar chart, and the specific combinations of data tables containing those proteins or genes are indicated with solid points below the bar chart. Total identifications for each data table are indicated on the right as ‘Set size’.

3.2 Sample-wise

The figures below show the number of proteins or genes identified/quantified (non-missing values) in each sample. Samples from different batches are coded with different shapes, and samples from different classes are coded with different colors. A separate figure is shown for each data table.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

3.3 Missing value distribution

The missing value distribution provides an overview of the completeness of the data. The table below shows the percent of missing values for all samples in each data table.

data table non_missing_value_ratio
diann 0.9022
diaumpire 0.9255
diaumpiredda 0.9208
msfraggerdia 0.9224
msfraggerdiadda 0.9190
tmt 0.9526

The following barplots show missing value distributions for each data table as number (Y axis)/percentage (number above bar) of proteins or genes with missing values in each bin. Genes are binned by proportion of samples with missing values from 0.1 to 1 in increments of 0.1, where 0.1 indicates missing values in no more than 10% of the samples, and 1 indicates missing values in all samples.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

4 Data normalization

4.1 Boxplot

Normalized data is expected to be centered around a similar value and show similar distributions in all samples. The boxplots below show the protein or gene expression measurement distribution across samples in each data table, allowing for qualitative assessment of the normalized data. Samples in input order are indicated on the X axis. The Y axis shows log2 transformed protein or gene values. Samples from different classes are coded with different colors.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

To quantify the normalization effect, we tested for how well the data in the feature set can distinguish between each pair of samples. If the distribution is similar for the two samples in a given pair, the overall feature abundance (levels for all features in one sample vs the other) should not be sufficient to predict which sample is which. Therefore, for each pair of samples, an AUROC test was performed to quantify the ability of feature abundance to distinguish the two samples, and then a data_dist_similarity score was generated: 1-2*abs(AUROC-0.5). This score ranges from 0 to 1, and the higher the score is the better the normalized data quality is (no systematic difference between the two samples). The final metric for each data table is the median of scores from all sample pairs. The column ‘n’ shows the total number of sample pairs in the analysis.

data table data_dist_similarity n
diann 0.9436 16471
diaumpire 0.9601 16471
diaumpiredda 0.9567 16471
msfraggerdia 0.9593 16471
msfraggerdiadda 0.9592 16471
tmt 0.9824 16471

4.2 Density plot

The density plots below show the expression distributions for all samples (separate line) in each data table. The Y axis shows the density over the range of log2 transformed protein or gene expression values (X axis).

5 Batch effect

5.1 Correlation heatmap

Another way to qualitatively assess batch effect is to visualize the correlations for measurements between samples from the same batch to those in samples from different batches using heatmaps. The following figures show Spearman correlation heatmaps for all pairs of samples (all samples included in both rows and columns) for each data table. The color indicates the correlation between samples. The samples are ordered by batches. Concentration of high correlation values (red color) for pairs of samples from the same batch block compared to other batches indicates the presence of batch effect.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

6 Biological signal

6.1 Correlation among protein complex members

Members of the same protein complex often show greater correlation in gene and protein expression (IntraComplex correlation) than genes or proteins that are in different complexes (InterComplex correlation). Thus, one way to evaluate the quality of the biological signal present in a data table is to compare IntraComplex correlation to InterComplex correlation. Furthermore, because of the need to preserve stoichiometry between protein complex members, the difference between IntraComplex correlation and InterComplex correlation is often greater at the protein level than at the RNA data. If both RNA and protein data tables are available, observing that this difference is more pronounced in the protein data table than the RNA data table serves as an indicator for the quality of the protein data. We use the protein complexes from the CORUM database in this analysis.

The boxplots below show the distributions and ranges for pairwise correlations between genes or proteins from the same complex and for genes and proteins from different complexes for each data table.

The table below shows a summary of the evaluation. ‘diff’ is Cor(intra) - Cor(inter). ‘complex_auc’ is the AUROC value based on correlation of protein pairs from different groups.

data table InterComplex IntraComplex diff complex_auc
diann 0.0244 0.2627 0.2384 0.7878
diaumpire 0.0414 0.2900 0.2485 0.7950
diaumpiredda 0.0429 0.2934 0.2505 0.7970
msfraggerdia 0.0398 0.2842 0.2443 0.7903
msfraggerdiadda 0.0363 0.2808 0.2445 0.7961
RNA 0.0213 0.1574 0.1361 0.6612
tmt 0.0259 0.4540 0.4281 0.8668

6.2 Gene function prediction

Previous studies have shown that expression correlation is often higher for functionally related genes or proteins than for unrelated genes or proteins and that this correlation is greater when considering protein data than when considering RNA data (Wang, Jing, et al. Molecular & Cellular Proteomics 16.1 (2017): 121-134.). Therefore, we can also evaluate the biological signal present in a data table by evaluating functional category predictions made using a co-expression network generated from each data table.

In this evaluation, each data table was used to build a co-expression network. For a selected network and a selected functional category (such as a selected category from GO or KEGG), proteins/genes annotated to the category and also included in the network were defined as a positive protein/gene set, and other proteins/genes in the network constituted the negative protein/gene set for the category. For a selected functional category, a subset of the proteins/genes were used as seed proteins/genes for random walk through the network to calculate scores for other proteins/genes. A higher score for a protein/gene represents a closer relationship between the protein/gene and the seed proteins/genes. The table below shows AUROCs of the prediction performance using this score for each selected functional category.

diann diaumpire diaumpiredda msfraggerdia msfraggerdiadda RNA tmt
Adherens junction 0.74 0.779 0.649 0.758 0.757 0.54 0.668
Adipocytokine signaling pathway 0.63 0.694 0.726 0.64 0.664 0.515 0.664
Alanine, aspartate and glutamate metabolism 0.699 0.669 0.716 0.603 0.646 0.667 0.813
Alzheimers disease 0.818 0.811 0.819 0.787 0.839 0.809 0.868
Amino sugar and nucleotide sugar metabolism 0.734 0.736 0.752 0.633 0.653 0.69 0.759
Amoebiasis 0.821 0.734 0.749 0.692 0.765 0.691 0.753
Amyotrophic lateral sclerosis (ALS) 0.652 0.636 0.627 0.57 0.672 0.614 0.748
Antigen processing and presentation 0.935 0.895 0.928 0.86 0.899 0.908 0.934
Apoptosis 0.715 0.589 0.641 0.581 0.577 0.642 0.712
Arachidonic acid metabolism 0.649 0.605 0.575 0.632 0.59 0.713 0.6
Arginine and proline metabolism 0.717 0.74 0.783 0.787 0.725 0.654 0.676
Arrhythmogenic right ventricular cardiomyopathy (ARVC) 0.744 0.793 0.78 0.748 0.675 0.613 0.709
Ascorbate and aldarate metabolism 0.915 0.85 0.851 0.749 0.79 0.863 0.905
Axon guidance 0.728 0.658 0.621 0.674 0.586 0.601 0.623
B cell receptor signaling pathway 0.721 0.715 0.779 0.707 0.792 0.57 0.85
Bacterial invasion of epithelial cells 0.654 0.735 0.764 0.708 0.726 0.631 0.783
beta-Alanine metabolism 0.72 0.784 0.748 0.754 0.679 0.659 0.74
Bile secretion 0.719 0.799 0.689 0.772 0.747 0.576 0.799
Butanoate metabolism 0.783 0.754 0.805 0.798 0.813 0.708 0.822
Calcium signaling pathway 0.631 0.706 0.585 0.584 0.765 0.597 0.715
Cardiac muscle contraction 0.897 0.902 0.846 0.919 0.848 0.708 0.881
Cell adhesion molecules (CAMs) 0.822 0.8 0.812 0.837 0.782 0.652 0.792
Cell cycle 0.735 0.752 0.836 0.742 0.772 0.745 0.822
Chagas disease (American trypanosomiasis) 0.78 0.781 0.649 0.64 0.626 0.548 0.729
Chemokine signaling pathway 0.784 0.784 0.775 0.669 0.786 0.678 0.791
Chronic myeloid leukemia 0.681 0.637 0.763 0.664 0.586 0.561 0.73
Citrate cycle (TCA cycle) 0.923 0.957 0.921 0.917 0.888 0.736 0.907
Collecting duct acid secretion 0.792 0.853 0.882 0.934 0.882 0.567 0.904
Colorectal cancer 0.633 0.756 0.694 0.64 0.728 0.653 0.694
Complement and coagulation cascades 0.953 0.915 0.93 0.95 0.972 0.702 0.99
Cysteine and methionine metabolism 0.727 0.675 0.578 0.736 0.762 0.605 0.75
Dilated cardiomyopathy 0.71 0.727 0.638 0.685 0.649 0.673 0.696
DNA replication 0.856 0.844 0.919 0.841 0.823 0.946 0.904
Drug metabolism - cytochrome P450 0.808 0.695 0.749 0.761 0.743 0.741 0.839
Drug metabolism - other enzymes 0.688 0.663 0.641 0.652 0.653 0.726 0.672
ECM-receptor interaction 0.854 0.877 0.909 0.899 0.903 0.758 0.856
Endocytosis 0.693 0.704 0.729 0.73 0.668 0.581 0.735
Epithelial cell signaling in Helicobacter pylori infection 0.709 0.828 0.783 0.818 0.769 0.609 0.753
ErbB signaling pathway 0.663 0.631 0.683 0.631 0.649 0.578 0.661
Fatty acid metabolism 0.885 0.867 0.874 0.853 0.796 0.744 0.854
Fc epsilon RI signaling pathway 0.763 0.931 0.724 0.712 0.789 0.672 0.785
Fc gamma R-mediated phagocytosis 0.808 0.76 0.767 0.779 0.769 0.668 0.776
Focal adhesion 0.766 0.792 0.832 0.757 0.793 0.747 0.79
Fructose and mannose metabolism 0.826 0.755 0.724 0.582 0.701 0.718 0.803
Galactose metabolism 0.74 0.737 0.729 0.705 0.683 0.754 0.578
Gap junction 0.637 0.742 0.717 0.623 0.691 0.57 0.607
Gastric acid secretion 0.578 0.558 0.614 0.593 0.634 0.601 0.713
Glutathione metabolism 0.61 0.586 0.507 0.573 0.665 0.611 0.566
Glycerolipid metabolism 0.66 0.676 0.697 0.664 0.682 0.652 0.763
Glycerophospholipid metabolism 0.646 0.674 0.627 0.715 0.598 0.659 0.78
Glycine, serine and threonine metabolism 0.761 0.715 0.719 0.671 0.774 0.752 0.775
Glycolysis / Gluconeogenesis 0.757 0.787 0.734 0.729 0.752 0.698 0.805
Glyoxylate and dicarboxylate metabolism 0.729 0.785 0.752 0.718 0.757 0.81 0.802
GnRH signaling pathway 0.674 0.814 0.616 0.627 0.751 0.592 0.696
Hematopoietic cell lineage 0.799 0.779 0.77 0.73 0.678 0.611 0.775
Hepatitis C 0.758 0.81 0.775 0.772 0.796 0.558 0.728
Histidine metabolism 0.711 0.83 0.857 0.816 0.649 0.697 0.776
Huntingtons disease 0.846 0.872 0.845 0.863 0.867 0.735 0.866
Hypertrophic cardiomyopathy (HCM) 0.76 0.815 0.69 0.777 0.715 0.589 0.74
Inositol phosphate metabolism 0.573 0.615 0.656 0.637 0.783 0.691 0.68
Insulin signaling pathway 0.636 0.689 0.654 0.643 0.692 0.564 0.702
Leishmaniasis 0.827 0.838 0.876 0.831 0.783 0.772 0.887
Leukocyte transendothelial migration 0.789 0.832 0.773 0.851 0.763 0.691 0.775
Long-term depression 0.699 0.749 0.658 0.701 0.747 0.589 0.772
Long-term potentiation 0.61 0.676 0.587 0.603 0.701 0.543 0.8
Lysine degradation 0.856 0.901 0.884 0.837 0.838 0.651 0.812
Lysosome 0.848 0.888 0.805 0.832 0.872 0.764 0.893
Malaria 0.776 0.662 0.713 0.858 0.8 0.7 0.79
MAPK signaling pathway 0.594 0.629 0.616 0.564 0.612 0.577 0.625
Melanogenesis 0.566 0.738 0.554 0.671 0.72 0.637 0.813
Metabolic pathways 0.719 0.732 0.746 0.734 0.735 0.655 0.785
Metabolism of xenobiotics by cytochrome P450 0.828 0.761 0.752 0.724 0.789 0.743 0.774
mRNA surveillance pathway 0.788 0.781 0.806 0.801 0.802 0.722 0.831
N-Glycan biosynthesis 0.933 0.97 0.918 0.944 0.884 0.689 0.916
Natural killer cell mediated cytotoxicity 0.718 0.704 0.692 0.657 0.764 0.744 0.783
Neurotrophin signaling pathway 0.749 0.715 0.657 0.635 0.7 0.617 0.645
NOD-like receptor signaling pathway 0.641 0.652 0.652 0.584 0.523 0.685 0.797
Oocyte meiosis 0.792 0.704 0.805 0.637 0.766 0.578 0.803
Osteoclast differentiation 0.652 0.665 0.659 0.711 0.752 0.622 0.811
Oxidative phosphorylation 0.919 0.974 0.949 0.974 0.967 0.863 0.958
Pancreatic cancer 0.624 0.757 0.682 0.699 0.576 0.603 0.633
Pancreatic secretion 0.61 0.582 0.573 0.648 0.672 0.583 0.813
Parkinsons disease 0.838 0.835 0.807 0.848 0.843 0.782 0.903
Pathogenic Escherichia coli infection 0.689 0.814 0.782 0.825 0.757 0.555 0.737
Pathways in cancer 0.706 0.683 0.646 0.604 0.643 0.555 0.57
Pentose and glucuronate interconversions 0.794 0.868 0.715 0.735 0.77 0.666 0.805
Pentose phosphate pathway 0.695 0.76 0.753 0.68 0.825 0.604 0.669
Peroxisome 0.777 0.818 0.806 0.831 0.766 0.814 0.878
Phagosome 0.823 0.855 0.847 0.892 0.813 0.693 0.849
Phosphatidylinositol signaling system 0.599 0.621 0.634 0.666 0.641 0.678 0.585
Porphyrin and chlorophyll metabolism 0.723 0.568 0.707 0.611 0.649 0.638 0.732
PPAR signaling pathway 0.716 0.733 0.783 0.767 0.709 0.69 0.705
Prion diseases 0.791 0.721 0.734 0.663 0.613 0.76 0.813
Propanoate metabolism 0.875 0.862 0.882 0.881 0.853 0.79 0.881
Prostate cancer 0.575 0.684 0.637 0.561 0.581 0.611 0.715
Proteasome 0.993 0.995 0.991 0.995 0.991 0.85 0.986
Protein digestion and absorption 0.869 0.814 0.881 0.803 0.783 0.712 0.912
Protein export 0.82 0.952 0.863 0.995 0.859 0.766 0.964
Protein processing in endoplasmic reticulum 0.778 0.798 0.767 0.759 0.744 0.731 0.826
Purine metabolism 0.699 0.655 0.633 0.667 0.679 0.627 0.658
Pyrimidine metabolism 0.621 0.6 0.615 0.624 0.574 0.612 0.596
Pyruvate metabolism 0.763 0.8 0.807 0.785 0.755 0.589 0.809
Regulation of actin cytoskeleton 0.627 0.768 0.759 0.756 0.734 0.714 0.781
Renal cell carcinoma 0.726 0.591 0.649 0.625 0.586 0.694 0.602
Retinol metabolism 0.716 0.745 0.793 0.701 0.854 0.716 0.907
Rheumatoid arthritis 0.819 0.861 0.88 0.827 0.874 0.771 0.932
Ribosome 0.967 0.98 0.954 0.96 0.986 0.913 0.985
Ribosome biogenesis in eukaryotes 0.84 0.861 0.847 0.746 0.804 0.731 0.826
RNA degradation 0.74 0.733 0.842 0.718 0.785 0.622 0.724
RNA transport 0.752 0.751 0.791 0.803 0.822 0.67 0.816
Salivary secretion 0.6 0.627 0.611 0.59 0.571 0.611 0.743
Shigellosis 0.71 0.782 0.75 0.789 0.686 0.789 0.846
Small cell lung cancer 0.645 0.859 0.804 0.748 0.72 0.64 0.745
SNARE interactions in vesicular transport 0.744 0.721 0.816 0.79 0.821 0.567 0.775
Spliceosome 0.895 0.931 0.898 0.856 0.898 0.798 0.864
Staphylococcus aureus infection 0.965 0.886 0.903 0.939 0.904 0.944 0.964
Starch and sucrose metabolism 0.734 0.827 0.806 0.717 0.803 0.578 0.82
Systemic lupus erythematosus 0.918 0.919 0.875 0.934 0.944 0.862 0.821
T cell receptor signaling pathway 0.66 0.678 0.716 0.584 0.726 0.697 0.714
TGF-beta signaling pathway 0.855 0.842 0.854 0.791 0.676 0.667 0.699
Tight junction 0.684 0.776 0.721 0.639 0.68 0.616 0.71
Toll-like receptor signaling pathway 0.633 0.541 0.594 0.577 0.583 0.576 0.719
Toxoplasmosis 0.77 0.736 0.735 0.674 0.683 0.588 0.738
Tryptophan metabolism 0.749 0.755 0.778 0.73 0.743 0.7 0.825
Tyrosine metabolism 0.712 0.593 0.69 0.567 0.713 0.792 0.715
Ubiquitin mediated proteolysis 0.76 0.567 0.72 0.69 0.677 0.65 0.75
Valine, leucine and isoleucine degradation 0.87 0.847 0.833 0.835 0.814 0.731 0.912
Vascular smooth muscle contraction 0.745 0.82 0.813 0.719 0.868 0.58 0.757
Vasopressin-regulated water reabsorption 0.656 0.727 0.759 0.669 0.777 0.649 0.787
VEGF signaling pathway 0.595 0.645 0.655 0.648 0.742 0.663 0.731
Vibrio cholerae infection 0.741 0.714 0.703 0.783 0.8 0.654 0.84
Viral myocarditis 0.858 0.877 0.832 0.828 0.836 0.85 0.713
Wnt signaling pathway 0.606 0.694 0.696 0.671 0.722 0.674 0.603

The rank boxplots below summarize the relative performance of the data tables in the functional prediction analysis. For each functional category, a rank is assigned to each data table based on its AUROC compared to the other data tables, where the best functional prediction rank is 1 and the poorest rank is the number of data tables.

Comparison of each protein (RNA) data table to a designated RNA (protein) data table is also summarized in the scatter plots below. For each point, the AUROC for a given category in the RNA data is plotted on the X-axis whereas the corresponding AUROC in the protein data table is plotted on the Y-axis. The number of categories for which the protein data table outperforms the RNA data table (AUROC(protein) > 1.1 * AUROC (RNA); red dots) and vice versa (AUROC(RNA) > 1.1 * AUROC (protein); blue dots) are also shown.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

6.3 PCA with sample class annotation

Another approach for assessing how well each data table can distinguish between classes is to determine how well each class can be separated by principal component analysis (PCA). In PCA score plots for each data table below, each point is a sample that is colored by class and that has a shape reflecting the batch. For a given sample, the PC2 score is plotted on the Y-axis whereas the PC1 score is plotted on the X-axis. Ellipses highlighting clusters of samples in each class are colored by corresponding class, and the separation between these ellipses indicates how well the variances captured by the first two PCs can distinguish between samples from different classes.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

6.4 Unsupervised clustering

Unsupervised hierarchical clustering can reveal patterns in the data (clusters of genes or samples that behave more similarly to each other than to other genes or samples). Each heatmap below shows the results of hierarchical clustering for a given data table using ComplexHeatmap. Genes/proteins are in rows, while samples are in columns and labeled with corresponding class to visualize any potential associations between classes and clusters.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

7 Multi-omics concordance

The concordance between the protein data and RNA data can be used to assess data quality when both RNA and protein data tables are available. Here, we evaluate gene- and sample-wise correlations between the protein and RNA data tables.

7.1 Gene-wise mRNA-protein correlation

The table below shows the number of genes with measurements (n) in each data table as well as the median of all gene-wise Spearman correlations between mRNA and protein measurements. The columns n5, n6, n7 and n8 show the number of genes with correlation greater than 0.5, 0.6, 0.7 and 0.8, respectively.

data table n n5 n6 n7 n8 gene_wise_cor
diann 5947 2013 1318 683 239 0.3820
diaumpire 6423 2143 1416 745 258 0.3733
diaumpiredda 6585 2129 1432 735 261 0.3675
msfraggerdia 6347 2088 1414 717 261 0.3683
msfraggerdiadda 6455 2095 1404 720 256 0.3693
tmt 9107 3366 2194 1198 459 0.4043

Spearman correlation results are also shown for each gene/protein in the boxplot below.

Another way to visualize the differences between the distributions of all gene-wise RNA-protein correlations is with the cumulative distribution function (CDF) plot shown below. Here each line shows the cumulative distribution for the gene-wise correlations. The further the distribution function is shifted to the right, the more highly correlated the RNA-protein data is.

The histograms below provide another way to visualize the distribution of correlations for each protein (or RNA) data table with the RNA (or protein) data. Here the bars showing binned frequencies of positive correlations are in red, while negative correlations are shown in the blue bins, and summary statistics are also provided.

dianndiaumpirediaumpireddamsfraggerdiamsfraggerdiaddatmt

7.2 Sample-wise mRNA-protein correlation

Sample-wise RNA-protein correlations are summarized in the table below as the median of Spearman correlations for matched protein and RNA data from all pairs of samples for each data table, while the violin plots below show the distributions of these correlations for each data table.

data table sample_wise_cor
diann 0.4025
diaumpire 0.3888
diaumpiredda 0.3914
msfraggerdia 0.3867
msfraggerdiadda 0.3887
tmt 0.4595